Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks

摘要

Factorization Machines (FMs) are a supervised learning approach that enhancesthe linear regression model by incorporating the second-order featureinteractions. Despite effectiveness, FM can be hindered by its modelling of allfeature interactions with the same weight, as not all feature interactions areequally useful and predictive. For example, the interactions with uselessfeatures may even introduce noises and adversely degrade the performance. Inthis work, we improve FM by discriminating the importance of different featureinteractions. We propose a novel model named Attentional Factorization Machine(AFM), which learns the importance of each feature interaction from data via aneural attention network. Extensive experiments on two real-world datasetsdemonstrate the effectiveness of AFM. Empirically, it is shown on regressiontask AFM betters FM with a $8.6\%$ relative improvement, and consistentlyoutperforms the state-of-the-art deep learning methods Wide&Deep and DeepCrosswith a much simpler structure and fewer model parameters. Our implementation ofAFM is publicly available at:https://github.com/hexiangnan/attentional_factorization_machine